Using crowdsourcing to improve sentiment analytics

ABSTRACT

A method and computer for managing analysis of sentiment is disclosed. A computer retrieves data used to perform the analysis of sentiment. The computer analyzes the data and the analysis of sentiment to determine if a gap exists requiring further processing to improve the analysis of sentiment. Responsive to a determination that the gap exists requiring further processing to improve the analysis of sentiment, the computer generates a task to address the gap. The computer then uses crowdsourcing to submit the generated task for processing.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The disclosure relates generally to using data processing systems forsentiment analytics and, in particular, to managing improvements forsentiment analytics. Still more particularly, the present disclosurerelates to using crowdsourcing to perform specified tasks associatedwith improving the effectiveness of sentiment analysis.

2. Description of the Related Art

Social media and Web 2.0 Technologies® have significantly enhancedinteractive information sharing and collaboration over the Internet.Social networking sites, such as Facebook® and Twitter providecollaboration tools which allow users to interact with each other byexchanging messages with other computer users. Typical collaborationtools include tools for chatting, texting, instant messaging, multimediamessaging, emailing, conferencing, tweeting, and commenting.

Through the use of collaboration tools users sometimes expresssentiments about features and aspects of entities. An entity issomething that exists by itself, although it need not be of materialexistence. An entity is defined as a particular and discrete unit. Forexample, an entity may be a person, an organization, a thing, anabstract idea, a problem, a solution, and a particular type of activity.The entities discussed may be particularly named persons, places, orthings and sometimes may include unnamed entities. An aspect of anentity is an appearance of the entity to the eye or mind. Some aspectsof entities include physical appearances such as those based on sight,touch, smell, and other senses. Other aspects of entities includedescriptions of metaphysical concepts, such as an aspect of a particularphilosophy. More particularly, entities may have aspects that contrastone entity from other entities.

Sentiment of an aspect of an entity is a subjective expression of apositive, negative or neutral opinion of the aspect of the entity. Forexample, sentiment of an aspect of an entity may be defined on a scalesuch as between −5 to +5 with −5 being the most negative, +5 the mostpositive, and 0 as neutral.

Sentiment analytics refers to the application of natural languageprocessing, computational linguistics, and text analytics to identifyand extract subjective information in source materials. Using sentimentanalytics, computer programs can mine source material to derive usersentiment about various aspects of entities. However, gaps may existwherein the sentiment analysis of the source materials is determined tobe insufficient to derive sentiment for some aspects of some entities.

Crowdsourcing is a process for performing certain kinds of tasks. In acrowdsourcing effort or procedure, a large group of organizations,individuals and other entities that desire to provide pertinentservices, such as a specific community of providers or the generalpublic, are invited to participate in a task that is presented by a taskrequester. At present, a crowdsourcing platform may serve as a broker orintermediary between the task requester and providers who are interestedin undertaking or participating in task performance. Crowdsourcingplatforms generally allow requesters to publish or broadcast theirchallenges and tasks, and further allow participating providers that aresuccessful in completing the task to receive specified monetary rewardsor other incentives. Innocentive®, TopCoder®, and MechanicalTurk® areexamples of presently available platforms.

Currently however, there is no system or process available for creatingand submitting crowdsourced tasks to address computer identified gaps insentiment analysis of source materials determined to be insufficient toderive sentiment for aspects of entities.

Therefore, it would be advantageous to have a method, data processingsystem, and computer program product that takes into account at leastsome of the issues discussed above, as well as possibly other issues.

In one illustrative embodiment, a method, data processing system, andcomputer program product for managing analysis of sentiment isdisclosed. A data processing system analyzes the data and the analysisof sentiment to determine if a gap exists requiring further processingto improve the analysis of sentiment. Responsive to a determination thatthe gap exists requiring further processing to improve the analysis ofsentiment, the data processing system generates a task to address thegap. The data processing system then uses crowdsourcing to submit thegenerated task for processing.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an illustration of a sentiment analytics managementenvironment for managing analysis of sentiment in accordance with anillustrative embodiment;

FIG. 2 is a block diagram of components involved in managing analysis ofsentiment in a data processing system in a sentiment analyticsmanagement system in accordance with an illustrative embodiment;

FIG. 3 is a flow chart of a process for managing analysis of sentimentin accordance with an illustrative embodiment;

FIG. 4 is a flow chart of a process for managing analysis of sentimentfor aspects of an entity corresponding to an activity in accordance withan illustrative embodiment;

FIG. 5 is a flow chart of a process for identifying user sentiment andgroup sentiment of social networks for aspects of entities in accordancewith an illustrative embodiment;

FIG. 6 is a flow chart of a process for generating and sending requestsfor additional sentiment information for aspects of entities ofcorresponding to an activity in accordance with an illustrativeembodiment; and

FIG. 7 is an illustration of a data processing system in accordance withan illustrative embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

As will be appreciated by one skilled in the art, aspects of the presentinvention may be embodied as a system, method or computer programproduct. Accordingly, aspects of the present invention may take the formof an entirely hardware embodiment, an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit,” “module” or “system.”Furthermore, aspects of the present invention may take the form of acomputer program product embodied in one or more computer readablemedium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing.

Computer program code for carrying out operations for aspects of thepresent invention may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java, Smalltalk, C++ or the like and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Aspects of the present invention are described below with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

The illustrative embodiments recognize and take into account thatexisting web based applications are available for sharing sentimentinformation of users. Web based applications typically run on a webserver. These web based applications communicate with web browsers andother applications by generating documents such as web pages and sendingthe documents and other data over a network. The generated web pages aretypically displayed on a display screen by a web browser running on acomputing device. For example, a web based application for acrowdsourcing platform may generate a web page for a user account for anowner of the user account. In this example, the owner of the useraccount may make selections for performing and managing tasks in thecrowdsourcing platform. For example, responsive to accepting a task, theowner of the user account may provide answers to questions whileperforming the task, such as by providing sentiments related to aspectsof entities.

As used herein, the term “computing device” means a hardware device witha processor unit and a capability to display information on a displaydevice and may also include the capability to emit audio on a speaker.For example, the computing device may be a computer, a television with aprocessor unit, a smart phone, and any other suitable device.

The illustrative embodiments also recognize and take into account thatexisting web based applications are available for sharing sentimentsabout aspects of entities with users of social networking web sites andother collaboration tools. For example, when a user is using a web basedapplication of a social network, the web based application may allow theuser to enter and share sentiments about topics of interest. Through theexpression of sentiment about the topics of interest the user mayprovide sentiments about particular aspects of entities associated withthe topic of interests. For example, the user may provide a positive ornegative opinion about a specific aspect of a specific entity.

The illustrative embodiments further recognize and take into accountthat existing sentiment analytics tools for performing natural languageprocessing, computational linguistics, and text analytics are availableto identify and extract subjective information in source materials.These sentiment analytics tools may be used to identify user sentimentabout aspects of entities from information entered by users into socialnetworks and collaboration tools.

Thus, the different illustrative embodiments provide a method,apparatus, and computer program product to manage analysis of sentiment.For example, the different illustrative embodiments may use a processfor managing analysis of sentiment. For example, a sentiment analyticsmanagement system may retrieve data used to perform an analysis ofsentiment for aspects of an entity corresponding to an activity. Thesentiment analytics management system may analyze the data and theanalysis of sentiment to determine if a gap exists requiring furtherprocessing to improve the analysis of sentiment. In this example, thegap corresponds to a topic about a particular aspect of the entity. Thesentiment analytics management system may generate a task to address thegap based on a template corresponding to a type of gap of the gap. Thesentiment analytics management system may also use crowdsourcing tosubmit the generated task for processing. After submitting the task forprocessing, the sentiment analytics management system may receive aresult of the processing of the task. Responsive to receiving a resultof the processing of the task, the sentiment analytics management systemmay then re-calculate the analysis of sentiment based on the result.

With reference now to the figures and, in particular, with reference toFIG. 1, an illustration of a sentiment analytics management environmentfor managing analysis of sentiment is depicted in accordance with anillustrative embodiment. In this illustrative example, sentimentanalytics management environment 100 is depicted in block diagram form.

In these illustrative examples, users, such as user 101, may use clientapplications running on computing devices such as client device 102within sentiment analytics management environment 100. Clientapplications running on client device 102 may take various forms. Forexample, client applications may include at least one of a web browser,a sentiment analytics management application, a database application, asmart-phone application, and other suitable types of applications forsentiment analytics management environment 100. In these illustrativeexamples, a web browser may display information on a display screen ofclient device 102 and may also present information to user 101 by usinga speaker to emit audio.

As used herein, the phrase “at least one of”, when used with a list ofitems, means that different combinations of one or more of the listeditems may be used and only one of each item in the list may be needed.For example, “at least one of item A, item B, and item C” may include,for example, without limitation, item A, or item A and item B. Thisexample also may include item A, item B, and item C, or item B and itemC. In other examples, “at least one of” may be, for example, withoutlimitation, two of item A, one of item B, and 10 of item C; four of itemB and seven of item C; and other suitable combinations.

As depicted, client device 102 communicates with sentiment analyticsmanagement system 106 over network 104. In these illustrative examples,user 101 may send activity 108 to sentiment analytics management system106 for processing. Activity 108 may be a development project, such asconstruction of cinemas, roads, utilities, government facilities,housing, commercial products, and any other type of activity for whichsentiment analysis is useful.

As also depicted, sentiment analytics management system 106 and clientdevice 102 may communicate with social network 110 and crowdsourcingplatform 112 over network 104. In these illustrative examples, socialnetwork 110 may be a web based application having data 114 entered byusers, such as user 101. As depicted, data 114 may comprise sentiments116 about aspects of entities associated with activities such asactivity 108. For example, one or more members of social network 110 mayhave previously entered sentiments 116 for one or more aspects ofentities corresponding to activity 108. Social network 110 may provideservices to search for, browse, enter, and modify data 114.

In these illustrative examples, crowdsourcing platform 112 serves as abroker or intermediary between task requesters and providers who areinterested in undertaking or participating in performance of tasksassociated with improving sentiment analytics for activities. Asdepicted, sentiment analytics management system 106 generates tasks andmakes requests to providers to process the tasks via crowdsourcingplatform 112. In these illustrative examples, crowdsourcing platform 112is used by sentiment analytics management system 106 in a process toimprove sentiment analysis for activity 108. For example, sentimentanalytics management system 106 may use crowdsourcing platform 112 topublish a task associated with resolving an issue with sentimentanalysis of a particular aspect of an entity corresponding to activity108. In this example, providers may perform the task and providecrowdsourcing information 118. For example, crowdsourcing information118 may include responses to surveys for additional sentiment and otherresponses to questions associated with resolving the issue with thesentiment analysis.

In these illustrative examples, domain ontology 120 is located withinsentiment analytics management environment 100. Domain ontology 120 mayinclude one or more aspects of one or more entities associated withactivity 108, one or more other activities related to activity 108,prioritization information associated with sentiment for each aspect andrelated activity, and other information suitable for use in identifyingtasks associated with activity 108. For example, domain ontology 120 maybe for a government domain, commercial domain and other suitabledomains. In this example domain ontology 120 may include a descriptionof targeted activities, such as cinema construction, government serviceconstruction, and other suitable activities. In these illustrativeexamples, domain ontology 120 may include descriptions of possibleissues related to the targeted activities such as traffic issues, safetyissues, environmental issues, service issues, and other suitable issues.Domain ontology 120 may also include descriptions of aspects of entitiesrelated to the activity, such as aspects of a location, aspects offacilities, aspect of organizations, and other suitable aspects ofentities. Domain ontology 120 may further include characteristics of atargeted audience, such as interest, profession, age, gender, and othersuitable characteristics of targeted audiences.

As depicted, sentiment analytics management system 106 may use domainontology 120 to identify tasks associated with improving sentimentanalysis for activity 108. For example, subsequent to analyzingsentiments 116 and domain ontology 120 sentiment analytics managementsystem 106 may identify an issue associated with sentiments 116 aboutaspects of entities in domain ontology 120. In this example, sentimentanalytics management system 106 may generate a task associated withresolving the issue and send the task to crowdsourcing platform 112 forprocessing. As another example, subsequent to analyzing sentiments 116and domain ontology 120 sentiment analytics management system 106 mayidentify an issue associated with information in domain ontology 120. Inthis example, a definition of an entity and/or aspect of an entity indomain ontology 120 may require refinement. As still another example,more training data may be required to improve the ability of sentimentanalytics management system 106 to analyze data 114 for identifyingsentiments 116.

In these illustrative examples, computer system 128 within group ofcomputer systems 126 is one or more computers. Also, as used herein,“group of” when used with reference to items means one or more items.For example, group of computer systems 126 is one or more computersystems. When computer system 128 is more than one computer, thecomputer system may take the form of a computer cluster, group ofcomputer systems 126, or other configurations of computer systemsconfigured to run server applications 130.

Sentiment analytics management system 106 uses sever applications 130located on computer system 128 in group of computer systems 126. Acomputer system is one or more computers in these illustrative examples.Server applications 130 may be any applications configured to processcommunications to provide services 132. Server applications 130 may be,for example, at least one of a database server, a file transfer protocol(FTP) server, a web server, a mail server, and/or other suitable typesof applications. Services 132 may include, for example, at least one ofretrieving a file, retrieving a webpage, retrieving information, writinginformation, downloading a program, and/or other suitable types ofaccess.

Sentiment analytics management system 106 may be implemented usinghardware, software, or a combination of the two. In these illustrativeexamples, sentiment analytics management system 106 is configured tomanage sentiment analysis using group of computer systems 126. In theseillustrative examples, sentiment analytics management system 106 isconfigured to use services 132 of server applications 130. For example,sentiment analytics management system 106 may use a service in services132 to perform sentiment analysis on data 114 and domain ontology 120for activity 108. As another example, sentiment analytics managementsystem 106 may use another service in services 132 to generate andpublish tasks, such as tasks identified by sentiment analyticsmanagement system 106 to improve the sentiment analysis. In theseillustrative examples, sentiment analytics management system 106 may beimplemented as a server, such as a web server, that is used by users formanaging sentiment analysis of activities.

The illustration of sentiment analytics management environment 100 inFIG. 1 is not meant to imply physical or architectural limitations tothe manner in which different illustrative embodiments may beimplemented. Other components in addition to and/or in place of the onesillustrated may be used. Some components may be unnecessary. Also, theblocks are presented to illustrate some functional components. One ormore of these blocks may be combined and/or divided into differentblocks when implemented in an illustrative embodiment.

For example, in some illustrative examples, additional social networks,in addition to social network 110, may be present within sentimentanalytics management environment 100. Also, although sentiment analyticsmanagement system 106 is shown as a block separate from group ofcomputer systems 126, sentiment analytics management system 106 may beimplemented in computer system 128. In other examples, sentimentanalytics management system 106 may be implemented in one or morecomputer systems in group of computer systems 126.

Further, although social network 110, crowdsourcing platform 112, andsentiment analytics management system 106 are shown in the block forsentiment analytics management environment 100 they may be remote fromeach other, such as by being in different locations. For example, socialnetwork 110 may be located in a first physical location andcrowdsourcing platform 112 may be located in a second physical locationdifferent from the first physical location.

Turning next to FIG. 2, a block diagram of components involved inmanaging analysis of sentiment in a data processing system in asentiment analytics management system is depicted in accordance with anillustrative embodiment. Sentiment analytics management system 200 is anexample of one implementation of sentiment analytics management system106 in sentiment analytics management environment 100 in FIG. 1.

In this illustrative example, data processing system 202 is present insentiment analytics management system 200. Data processing system 202may be any combination of hardware components implementing sentimentanalytics management system 200. Data processing system 202 comprisessentiment analytics management services 204. In these illustrativeexamples, sentiment analytics management services 204 comprise a numberof services that manage analysis of sentiment. Preferences management206 is a service included in sentiment analytics management services 204that manages user profiles of users of sentiment analytics managementsystem 200.

As depicted, preference profiles 208 include one or more profiles insentiment analytics management system 200 of users of sentimentanalytics management system 200. In these illustrative examples, profile210 is profile of a user located in preference profiles 208. Eachprofile in preference profiles 208 is an account or other record ofinformation for users of sentiment analytics management system 200. Inthese illustrated examples, profiles, such as profile 210, includepreferences 212 of users of sentiment analytics management system 200.Preferences 212 of users may be selected by a user, an administrator,and by a rule using default preferences. Preferences 212 includeintegration preferences 214 for using social networks and crowdsourcingplatforms. Preferences 212 may also include polices 216 for determiningif further processing is required to improve an analysis of sentimentinformation and for determining who to send tasks to address gaps insentiment analysis too. In these illustrative examples, policies 216 mayspecify aspects of entities for which sentiment information must meetparticular thresholds. The policies may include a threshold for aquality of sentiment information about an aspect of an entity, thresholdfor a distribution of sentiment information about an aspect of anentity, an absence of sentiment information about an aspect of anentity, and other suitable thresholds for determining if sentimentinformation is sufficient for an analysis of sentiment.

In these illustrative examples, integration preferences 214 in profile210 include an identification of one or more servers, such as server218. Integration preferences 214 may also include an identification ofinformation for communicating with server 218. For example, theinformation for communicating with server 218 may, includeidentification of networking protocols used by server 218 over anetwork, such as network 220, identification of networking addresses ofserver 218 in network 220, identification of user names and passwordsfor gaining access to data 222 in server 218, and any other suitableinformation for communicating with server 218 in sentiment analyticsmanagement system 200.

In these illustrative examples, server 218 may host one or more websites, such as social network 110 and crowdsourcing platform 112 inFIG. 1. As depicted, server 218 may include data 222, such as sentimentinformation 224 and social network 226. In these illustrative examples,sentiment information 224 and social network 226 may be used to performan analysis of sentiment. Sentiment information 224 is an example ofsentiments 116 in FIG. 1. In these illustrative examples, sentimentinformation 224 may comprise one or more sentiments about aspects ofentities. For example, the sentiments in sentiment information 224 maycomprise subjective expressions of positive, negative, and neutralopinions about the aspects of the entities. Social network 226 is anexample of a social network of a user of server 218, such as user 101 inFIG. 1 retrieved from data 114 in FIG. 1. In these illustrativeexamples, social network 226 may include identification ofrelationships, memberships, known instant messaging identifiers, knownemail identifiers, known phone numbers, corporate associations, legalassociations and any other suitable social networking information of auser of server 218 in sentiment analytics management system 200.

Sentiment analytics 228 is a service included in sentiment analyticsmanagement services 204 for generating an analysis of sentiment fromsentiment information 224. In these illustrative examples, sentimentanalytics 228 retrieves aspects 236 of entities 234 from domain ontology230 to generate analysis of sentiment 232. Domain ontology 230 is anexample of domain ontology 120 in FIG. 1. As depicted, sentimentanalytics 228 determines measures of sentiment 238 from data 222.Measures of sentiment 238 may include quality 240 of sentimentinformation 224, distribution 242 of sentiment information 224, and anyother suitable measure of sentiment in sentiment analytics managementsystem 200. In these illustrative examples, quality 240 of sentimentinformation 224 may include one or more of a count of a number of usersexpressing a same sentiment, a confidence level that sentiment expressedis meaningful, a number of members of a group agreeing with a sentiment,a weighted value indicating the strength of each sentiment, anassociation of the sentiment information with users meetingcharacteristics of a targeted audience, and any other suitable qualityfor sentiment information about aspects of entities.

Gap identification 243 is a service included in sentiment analyticsmanagement services 204 for identifying gaps, such as gap 244 in whichfurther processing is required to improve analysis of sentiment 232. Inthese illustrative examples, gap identification 243 uses policies 216 toidentify gap 244 based on measures of sentiment not meeting one or morethresholds in polices 216.

Crowdsourcing task management 248 is a service included in sentimentanalytics management services 204 for managing tasks to improve analysisof sentiment 232. As depicted, crowdsourcing task management 248generates task 250 from a task template matching the type of gap of gap244, such as task template 256 in task templates 254. In theseillustrative examples, task 250 comprises specification 252 forperforming task 250 generated by crowdsourcing task management 248.Specification 252 may comprise a description of activity 246, domainontology 230, a description of gap 244, a description of task 250,sentiment information 224, measures of sentiment 238 about aspects 236of entities 234 associated with gap 244, questions for a survey fromtask template 256, and other suitable instructions for completing task250 to address gap 244.

In these illustrative examples, crowdsourcing task management 248identifies one or more entities to perform task 250, such as crowdsource258. As depicted crowdsourcing task management 248 may identifycrowdsource 258 from users 262 based on sentiment 264 interests 266, andactivity 268 of each user in users 262. For example, crowdsourcing taskmanagement 248 may select crowdsource 258 based on a user in users 262having sentiment 264, interests 266, and activity 268 that meet a policyin policies 216 for selecting a user to perform a task associated with agap in analysis of sentiment 232. As depicted, activity 268 is a historyof sentiment entered by each user and a history of tasks performed byeach user. In these illustrative examples, sentiment 264 is an exampleof sentiment previously identified by a user of a crowdsourcingplatform, such as in previous activity 268. As depicted, interests 266identify aspects of entities for which a user is an expert. Interests266 may be determined by crowdsourcing task management 248 based onsentiment 264 and activity 268. Interests 266 may also have beenpreviously selected the user.

Crowdsourcing task management 248 sends generated task 250 tocrowdsource 258 and receives task response 270 from crowdsource 258. Inthese illustrative examples, task response 270 may comprise a result ofthe processing of task 250, such as additional sentiment informationother than sentiment information 224. Subsequent to receiving taskresponse 270, crowdsourcing task management 248 may add the result ofthe processing of the task returned in task response 270 to data 222.Crowdsourcing task management 248 may also record the response inactivity 268. Crowdsourcing task management 248 may further record anysentiment information provided in task response 270 to sentiment 264.

In these illustrative examples, sentiment analytics 228 may thenre-calculate analysis of sentiment 232 based in part on the result ofthe processing of the task returned in task response 270. In theseillustrative examples, gap identification 243 may also be used todetermine if the result returned in task response 270 has closed gap 244based on analysis of sentiment 232. For example, gap 244 may bedetermined to be closed when a policy in policies 216 that identifiedgap 244 in a previous analysis of sentiment no longer identifies gap244.

The illustration of sentiment analytics management system 200 in FIG. 2is not meant to imply physical or architectural limitations to themanner in which different illustrative embodiments may be implemented.Other components in addition to and/or in place of the ones illustratedmay be used. Some components may be unnecessary. Also, the blocks arepresented to illustrate some functional components. One or more of theseblocks may be combined and/or divided into different blocks whenimplemented in an illustrative embodiment.

For example, in some illustrative examples, additional services, inaddition to the services shown sentiment analytics management services204, may be present within sentiment analytics management system 200.For example, a service in services 132 in FIG. 1 to authorize computingdevices such as client device 102 in FIG. 1 may be included in sentimentanalytics management system 200. In this example, the authorization ofclient device 102 may be required by a policy in policies 216 beforeclient device 102 is allowed access to preference profiles 208, activity246, analysis of sentiment 232, task 250, task response 270, domainontology 230, task templates 254, and other information in sentimentanalytics management system 200.

As another example, a service in sentiment analytics management services204 in FIG. 2 to retrieve sentiment analytics information, such asdomain ontology 230, task templates 254, and training data for sentimentanalytics 228, may be included in sentiment analytics management system200. In this example, the service to retrieve sentiment analyticsinformation may execute on a selected intermittent basis or based onidentification of an issue in analysis of sentiment 232. Further in thisexample, the service to retrieve sentiment analytics information mayalso execute responsive to a notification from server 218 orcrowdsources 260 that a change has occurred in sentiment analyticsinformation that was previously retrieved by sentiment analyticsmanagement system 200.

Turning next to FIG. 3, an illustrative example of a flowchart of aprocess for managing analysis of sentiment is depicted in accordancewith an illustrative embodiment. The steps in FIG. 3 may be implementedin sentiment analytics management system 200 in FIG. 2. In particular,the steps in the process may be implemented in software, hardware, or acombination of the two in sentiment analytics management services 204 indata processing system 202 in FIG. 2.

The process begins by retrieving data used to perform an analysis ofsentiment (step 300). In this illustrated process, the data is anexample of data 222 in FIG. 2 and the analysis of sentiment is anexample of analysis of sentiment 232 in FIG. 2 that is generated bysentiment analytics 228 in FIG. 2.

The process next analyzes the data and the analysis of sentiment todetermine if a gap exists requiring further processing to improve theanalysis of sentiment (step 302). In this illustrated process, the gapis an example of gap 244 in FIG. 2 that is determined by gapidentification 243 in FIG. 2.

As depicted in step 304, if the gap does not exist, the processterminates. Otherwise, if the gap does exist, the process continues bygenerating a task to address the gap (step 306). In this illustratedprocess, the task is an example of task 250 in FIG. 2 that is generatedby crowd source task management 248 in FIG. 2.

The process then uses crowdsourcing, such as by using a crowdsourcingplatform, to submit the generated task for processing (step 308) withthe process terminating thereafter. In this illustrated process, thecrowdsourcing platform is an example of crowdsourcing platform 112 inFIG. 1.

Turning next to FIG. 4, an illustrative example of a flowchart of aprocess for managing analysis of sentiment for aspects of an entitycorresponding to an activity is depicted in accordance with anillustrative embodiment. The steps in FIG. 4 may be implemented insentiment analytics management system 200 in FIG. 2. In particular, thesteps in the process may be implemented in software, hardware, or acombination of the two in sentiment analytics management services 204 indata processing system 202 in FIG. 2.

The process begins by retrieving data used to perform an analysis ofsentiment for aspects of an entity corresponding to an activity (step400). In this illustrated process, the data is an example of data 222 inFIG. 2, the analysis of sentiment is an example of analysis of sentiment232 in FIG. 2, and the aspects of entities are examples of aspects 236of entities 234 in FIG. 2.

The process next analyzes the data and the analysis of sentiment todetermine if a gap exists requiring further processing to improve theanalysis of sentiment, wherein the gap corresponds to a topic about aparticular aspect of the entity (step 402). In this illustrated process,the gap is an example of gap 244 in FIG. 2 that is determined by gapidentification 243 in FIG. 2.

As illustrated in step 404, if the gap does not exist, the processterminates. Otherwise, if the gap does exist, the process continues bygenerating a task to address the gap based on a template correspondingto a type of gap of the gap (step 406). The process then usescrowdsourcing to submit the generated task for processing (step 408).

After submitting the task for processing, the process receives a resultof the processing of the task (step 410). The process then re-calculatesthe analysis of sentiment based on the result (step 412) with theprocess continuing to step 402.

Turning next to FIG. 5, an illustrative example of a flowchart of aprocess for identifying user sentiment and group sentiment of socialnetworks for aspects of entities is depicted in accordance with anillustrative embodiment. The steps in FIG. 5 may be implemented insentiment analytics management system 200 in FIG. 2. In particular, thesteps in the process may be implemented in software, hardware, or acombination of the two in sentiment analytics management services 204 indata processing system 202 in FIG. 2.

The process begins by retrieving data from sources of information thatmay contain user sentiments about aspects of entities, the sourcesincluding forums, blogs, social networking web sites, collaborationtools, and other suitable sources (step 500). The process nextidentifies a user associated with an entity based on an amount ofsentiment information entered by the user in the retrieved data (step502). The process retrieves social networking information about the useridentifying groups of users associated with the user (step 504). Theprocess uses the social networking information about the user and theretrieved data, to determine a group sentiment about the entity for thesocial network of the user (step 506). The process then stores the user,the social networking information about the user, the entity, thesentiment information, and the group sentiment in a repository (step508) with the process terminating thereafter.

Turning next to FIG. 6, an illustrative example of a flowchart of aprocess for generating and sending requests for additional sentimentinformation for aspects of entities of corresponding to an activity isdepicted in accordance with an illustrative embodiment. The steps inFIG. 6 may be implemented in sentiment analytics management system 200in FIG. 2. In particular, the steps in the process may be implemented insoftware, hardware, or a combination of the two in sentiment analyticsmanagement services 204 in data processing system 202 in FIG. 2.

The process begins by identifying aspects of an entity corresponding toan activity by using an ontology that describes entities and activitiesassociated with the activity (step 600). The process next selects a setof aspects from the aspects of the entity for which sentimentinformation fails to meet a threshold for one of quality of sentimentinformation about the respective aspect, distribution of sentimentinformation about the respective aspect, and absence of sentimentinformation about the respective aspect (step 602). The process thengenerates and sends a request for additional sentiment information foreach aspect in the set of selected aspects (step 604) with the processterminating thereafter.

Turning now to FIG. 7, an illustration of a data processing system isdepicted in accordance with an illustrative embodiment. Data processingsystem 700 is an example of a data processing system that may be used toimplement managing authorizations to access personal data in a sentimentanalytics management system. Data processing system 700 is also anexample of a data processing system that may be used to implementsentiment analytics management system 106 in FIG. 1 and computer system128 in FIG. 1. More particularly, data processing system 700 may be usedto implement data processing system 202 in FIG. 2 and server 218 in FIG.2. In this illustrative example, data processing system 700 includescommunications framework 702, which provides communications betweenprocessor unit 704, memory 706, persistent storage 708, communicationsunit 710, input/output (I/O) unit 712, and display 714. In theseexamples, communications frame work 704 may be a bus system.

Processor unit 704 serves to execute instructions for software that maybe loaded into memory 706. Processor unit 704 may be a number ofprocessors, a multi-processor core, or some other type of processor,depending on the particular implementation. A number, as used hereinwith reference to an item, means one or more items. Further, processorunit 704 may be implemented using a number of heterogeneous processorsystems in which a main processor is present with secondary processorson a single chip. As another illustrative example, processor unit 704may be a symmetric multi-processor system containing multiple processorsof the same type.

Memory 706 and persistent storage 708 are examples of storage devices716. A storage device is any piece of hardware that is capable ofstoring information, such as, for example, without limitation, data,program code in functional form, and/or other suitable informationeither on a temporary basis and/or a permanent basis. Storage devices716 may also be referred to as computer readable storage devices inthese examples. Memory 706, in these examples, may be, for example, arandom access memory or any other suitable volatile or non-volatilestorage device. Persistent storage 708 may take various forms, dependingon the particular implementation.

For example, persistent storage 708 may contain one or more componentsor devices. For example, persistent storage 708 may be a hard drive, aflash memory, a rewritable optical disk, a rewritable magnetic tape, orsome combination of the above. The media used by persistent storage 708also may be removable. For example, a removable hard drive may be usedfor persistent storage 708.

Communications unit 710, in these examples, provides for communicationswith other data processing systems or devices. In these examples,communications unit 710 is a network interface card. Communications unit710 may provide communications through the use of either or bothphysical and wireless communications links.

Input/output unit 712 allows for input and output of data with otherdevices that may be connected to data processing system 700. Forexample, input/output unit 712 may provide a connection for user inputthrough a keyboard, a mouse, and/or some other suitable input device.Further, input/output unit 712 may send output to a printer. Display 714provides a mechanism to display information to a user.

Instructions for the operating system, applications, and/or programs maybe located in storage devices 716, which are in communication withprocessor unit 704 through communications framework 702. In theseillustrative examples, the instructions are in a functional form onpersistent storage 708. These instructions may be loaded into memory 706for execution by processor unit 704. The processes of the differentembodiments may be performed by processor unit 704 using computerimplemented instructions, which may be located in a memory, such asmemory 706.

These instructions are referred to as program code, computer usableprogram code, or computer readable program code that may be read andexecuted by a processor in processor unit 704. The program code in thedifferent embodiments may be embodied on different physical or computerreadable storage media, such as memory 706 or persistent storage 708.

Program code 718 is located in a functional form on computer readablemedia 720 that is selectively removable and may be loaded onto ortransferred to data processing system 700 for execution by processorunit 704. Program code 718 and computer readable media 720 form computerprogram product 722 in these examples. In one example, computer readablemedia 720 may be computer readable storage media 724 or computerreadable signal media 726. Computer readable storage media 724 mayinclude, for example, an optical or magnetic disk that is inserted orplaced into a drive or other device that is part of persistent storage708 for transfer onto a storage device, such as a hard drive, that ispart of persistent storage 708. Computer readable storage media 724 alsomay take the form of a persistent storage, such as a hard drive, a thumbdrive, or a flash memory, that is connected to data processing system700. In some instances, computer readable storage media 724 may not beremovable from data processing system 700. In these examples, computerreadable storage media 724 is a physical or tangible storage device usedto store program code 718 rather than a medium that propagates ortransmits program code 718. Computer readable storage media 724 is alsoreferred to as a computer readable tangible storage device or a computerreadable physical storage device. In other words, computer readablestorage media 724 is a media that can be touched by a person.

Alternatively, program code 718 may be transferred to data processingsystem 700 using computer readable signal media 726. Computer readablesignal media 726 may be, for example, a propagated data signalcontaining program code 718. For example, computer readable signal media726 may be an electromagnetic signal, an optical signal, and/or anyother suitable type of signal. These signals may be transmitted overcommunications links, such as wireless communications links, opticalfiber cable, coaxial cable, a wire, and/or any other suitable type ofcommunications link. In other words, the communications link and/or theconnection may be physical or wireless in the illustrative examples.

In some illustrative embodiments, program code 718 may be downloadedover a network to persistent storage 708 from another device or dataprocessing system through computer readable signal media 726 for usewithin data processing system 700. For instance, program code stored ina computer readable storage medium in a server data processing systemmay be downloaded over a network from the server to data processingsystem 700. The data processing system providing program code 718 may bea server computer, a client computer, or some other device capable ofstoring and transmitting program code 718.

The different components illustrated for data processing system 700 arenot meant to provide architectural limitations to the manner in whichdifferent embodiments may be implemented. The different illustrativeembodiments may be implemented in a data processing system includingcomponents in addition to or in place of those illustrated for dataprocessing system 700. Other components shown in FIG. 7 can be variedfrom the illustrative examples shown. The different embodiments may beimplemented using any hardware device or system capable of runningprogram code. As one example, the data processing system may includeorganic components integrated with inorganic components and/or may becomprised entirely of organic components excluding a human being. Forexample, a storage device may be comprised of an organic semiconductor.

In another illustrative example, processor unit 704 may take the form ofa hardware unit that has circuits that are manufactured or configuredfor a particular use. This type of hardware may perform operationswithout needing program code to be loaded into a memory from a storagedevice to be configured to perform the operations.

For example, when processor unit 704 takes the form of a hardware unit,processor unit 704 may be a circuit system, an application specificintegrated circuit (ASIC), a programmable logic device, or some othersuitable type of hardware configured to perform a number of operations.With a programmable logic device, the device is configured to performthe number of operations. The device may be reconfigured at a later timeor may be permanently configured to perform the number of operations.Examples of programmable logic devices include, for example, aprogrammable logic array, a programmable array logic, a fieldprogrammable logic array, a field programmable gate array, and othersuitable hardware devices. With this type of implementation, programcode 718 may be omitted because the processes for the differentembodiments are implemented in a hardware unit.

In still another illustrative example, processor unit 704 may beimplemented using a combination of processors found in computers andhardware units. Processor unit 704 may have a number of hardware unitsand a number of processors that are configured to run program code 718.With this depicted example, some of the processes may be implemented inthe number of hardware units, while other processes may be implementedin the number of processors.

In another example, a bus system may be used to implement communicationsframework 702 and may be comprised of one or more buses, such as asystem bus or an input/output bus. Of course, the bus system may beimplemented using any suitable type of architecture that provides for atransfer of data between different components or devices attached to thebus system.

Additionally, a communications unit may include a number of more devicesthat transmit data, receive data, or transmit and receive data. Acommunications unit may be, for example, a modem or a network adapter,two network adapters, or some combination thereof. Further, a memory maybe, for example, memory 706, or a cache, such as found in an interfaceand memory controller hub that may be present in communicationsframework 702.

Thus, the illustrative embodiments provide a method, data processingsystem, and computer program product for managing analysis of sentiment.In one example, a program runs in a computer system and manages analysisof sentiment by generating and sending tasks to address gaps identifiedin the analysis of sentiment. In still another example, the program usescrowdsourcing to identify providers who are interested in undertaking orparticipating in performance of the tasks to address the gaps identifiedin the analysis of sentiment.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a,” “an,” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components but do not preclude the presence or addition of one ormore other features, integers, steps, operations, elements, components,and/or groups thereof.

The flowcharts and block diagrams in the figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowcharts or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be performed substantiallyconcurrently, or the blocks may sometimes be performed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustrations,and combinations of blocks in the block diagrams and/or flowchartillustrations, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below are intended toinclude any structure, material, or act for performing the function incombination with other claimed elements as specifically claimed. Thedescription of the present invention has been presented for purposes ofillustration and description, but is not intended to be exhaustive orlimited to the invention in the form disclosed. Many modifications andvariations will be apparent to those of ordinary skill in the artwithout departing from the scope and spirit of the invention. Theembodiment was chosen and described in order to best explain theprinciples of the invention and the practical application, and to enableothers of ordinary skill in the art to understand the invention forvarious embodiments with various modifications as are suited to theparticular use contemplated.

What is claimed is:
 1. A method for managing analysis of sentiment, themethod comprising: retrieving, by a computer, data used to perform theanalysis of sentiment; analyzing, by the computer, the data and theanalysis of sentiment using user preferences comprising integrationpreferences for using social networks and crowdsourcing platforms andpolicies to determine if a gap in the analysis of sentiment existsrequiring further processing to improve the analysis of sentiment,wherein the gap exists in sentiment analysis of source materialsdetermined to be insufficient to derive sentiment for aspects ofentities, and wherein gap identification uses the policies to identifythe gap based on measures of sentiment not meeting one or morethresholds in the policies; responsive to the computer analyzing thedata and the analysis of sentiment to determine if the gap in theanalysis of sentiment exists, determining, by the computer, an amount ofpositive sentiment in the data and negative sentiment in the data foreach topic in a set of topics corresponding to aspects of an entity andselecting, by the computer, a topic from the set of topics correspondingto a particular aspect of the entity for the further processing based ondetermining that the amount of positive sentiment in the data andnegative sentiment in the data for the selected topic fails to meet athreshold amount of positive and negative sentiment for the selectedtopic to derive sentiment for the particular aspect of the entity;generating, by the computer, a task using a particular task template, ina set of task templates, matching a type of gap to address the gap inthe analysis of sentiment regarding the selected topic that failed tomeet the threshold amount of positive and negative sentiment for theselected topic to derive sentiment for the particular aspect of theentity, wherein the task comprises a specification for performing thetask comprising a description of activity, a domain ontology, adescription of the gap, a description of the task, sentimentinformation, measures of sentiment about aspects of entities associatedwith the gap, questions for a survey from the task template, andinstructions for completing the task to address the gap; and using, bythe computer, crowdsourcing to submit the task for processing to addressthe gap in the analysis of sentiment regarding the selected topic thatfailed to meet the threshold amount of positive and negative sentimentfor the selected topic to derive sentiment for the particular aspect ofthe entity, wherein the computer using the crowdsourcing to submit thetask for processing further comprises: identifying, by the computer,users associated with the topic about the particular aspect of theentity based on an amount of sentiment information about the topic inretrieved data from each user; determining, by the computer, a groupsentiment about the topic corresponding to a group of users on a socialnetwork; and selecting, by the computer, one or more users within thegroup of users on the social network to submit the task to based on thedetermined group sentiment about the topic.
 2. The method of claim 1,further comprising: re-calculating, by the computer, the analysis ofsentiment based on receiving a result of the processing of the task toaddress the gap in the analysis of sentiment regarding the selectedtopic that failed to meet the threshold amount of positive and negativesentiment for the selected topic to derive sentiment for the particularaspect of the entity; and determining, by the computer, that thereceived result of the processing of the task closed the gap in theanalysis of sentiment regarding the selected topic based on there-calculating of the analysis of sentiment in response to a policy inthe policies that previously identified the gap no longer identifyingthe gap.
 3. The method of claim 1, wherein the analyzing of thesentiment is for the aspects of the entity corresponding to an activity,and wherein the gap in the analysis of sentiment requiring the furtherprocessing corresponds to the topic about the particular aspect of theentity.
 4. The method of claim 3, further comprising: identifying, bythe computer, the topic about the particular aspect of the entity basedon sentiment information about the topic in the data.
 5. The method ofclaim 4, wherein identifying, by the computer, the topic about theparticular aspect of the entity based on sentiment information about thetopic in the data comprises: determining, by the computer, a quality ofsentiment information in the data for each topic in the set of topics;and selecting, by the computer, the topic based on the quality ofsentiment information for the topic not meeting a threshold for qualityof sentiment information.
 6. The method of claim 3, further comprising:identifying, by the computer, the topic about the particular aspect ofthe entity based on an absence of information about the particularaspect of the entity in the data.
 7. The method of claim 1, wherein thetask generated by the computer comprises a specification for addressingthe gap based on the particular task template corresponding to the typeof gap of the gap.
 8. The method of claim 3, further comprising:identifying, by the computer, the aspects of the entity corresponding tothe activity by using an ontology that describes entities and activitiesassociated with the activity.
 9. A computer system for managing analysisof sentiment, the computer system comprising: a bus system; one or morecomputer readable storage devices connected to the bus system, whereinthe one or more computer readable storage devices store programinstructions; and one or more processors connected to the bus system,wherein the one or more processors execute the program instructions to:retrieve data used to perform the analysis of sentiment; analyze thedata and the analysis of sentiment using user preferences comprisingintegration preferences for using social networks and crowdsourcingplatforms and policies to determine if a gap in the analysis ofsentiment exists requiring further processing to improve the analysis ofsentiment, wherein the gap exists in sentiment analysis of sourcematerials determined to be insufficient to derive sentiment for aspectsof entities, and wherein gap identification uses the policies toidentify the gap based on measures of sentiment not meeting one or morethresholds in the policies; determine an amount of positive sentiment inthe data and negative sentiment in the data for each topic in a set oftopics corresponding to aspects of an entity and selecting, by thecomputer, a topic from the set of topics corresponding to a particularaspect of the entity for the further processing based on determiningthat the amount of positive sentiment in the data and negative sentimentin the data for the selected topic fails to meet a threshold amount ofpositive and negative sentiment for the selected topic to derivesentiment for the particular aspect of the entity in response toanalyzing the data and the analysis of sentiment to determine if the gapin the analysis of sentiment exists; generate a task using a particulartask template, in a set of task templates, matching a type of the gap toaddress the gap in the analysis of sentiment regarding the selectedtopic that failed to meet the threshold amount of positive and negativesentiment for the selected topic to derive sentiment for the particularaspect of the entity, wherein the task comprises a specification forperforming the task comprising a description of activity, a domainontology, a description of the gap, a description of the task, sentimentinformation, measures of sentiment about aspects of entities associatedwith the gap, questions for a survey from the task template, andinstructions for completing the task to address the gap; and usecrowdsourcing to submit the task for processing to address the gap inthe analysis of sentiment regarding the selected topic that failed tomeet the threshold amount of positive and negative sentiment for theselected topic to derive sentiment for the particular aspect of theentity, wherein using the crowdsourcing to submit the task forprocessing further comprises the one or more processors furtherexecuting the program instructions to: identify users associated withthe topic about the particular aspect of the entity based on an amountof sentiment information about the topic in retrieved data from eachuser; determine a group sentiment about the topic corresponding to agroup of users on a social network; and select one or more users withinthe group of users on the social network to submit the task to based onthe determined group sentiment about the topic.
 10. The computer systemof claim 9, wherein the one or more processors further execute theprogram instructions to: re-calculate the analysis of sentiment based onreceiving a result of the processing of the task to address the gap inthe analysis of sentiment regarding the selected topic that failed tomeet the threshold amount of positive and negative sentiment for theselected topic to derive sentiment for the particular aspect of theentity; and determine that the received result of the processing of thetask closed the gap in the analysis of sentiment regarding the selectedtopic based on re-calculating of the analysis of sentiment in responseto a policy in the policies that previously identified the gap no longeridentifying the gap.
 11. The computer system of claim 9, whereinanalyzing of the sentiment is for the aspects of the entitycorresponding to an activity, and wherein the gap in the analysis ofsentiment requiring the further processing corresponds to the topicabout the particular aspect of the entity.
 12. The computer system ofclaim 11, wherein the one or more processors further execute the programinstructions to: identify the topic about the particular aspect of theentity based on sentiment information about the topic in the data. 13.The computer system of claim 12, wherein the program instructionsexecuted by the one or more processors to identify the topic about theparticular aspect of the entity based on sentiment information about thetopic in the data are further executed by the one or more processors to:determine a quality of sentiment information in the data for each topicin the set of topics; and select the topic based on the quality ofsentiment information for the topic not meeting a threshold for qualityof sentiment information.
 14. The computer system of claim 11, whereinthe one or more processors further execute the program instructions to:identify the topic about the particular aspect of the entity based on anabsence of information about the particular aspect of the entity in thedata.
 15. The computer system of claim 9, wherein the task comprises aspecification for addressing the gap based on the particular tasktemplate corresponding to the type of gap of the gap.
 16. The computersystem of claim 11, wherein the one or more processors further executethe program instructions to: identify the aspects of the entitycorresponding to the activity by using an ontology that describesentities and activities associated with the activity.
 17. A computerprogram product for managing analysis of sentiment, the computer programproduct comprising a computer readable storage device having programinstructions embodied therewith, the program instructions executable bya computer to cause the computer to perform a method comprising:retrieving, by the computer, data used to perform the analysis ofsentiment; analyzing, by the computer, the data and the analysis ofsentiment using user preferences comprising integration preferences forusing social networks and crowdsourcing platforms and policies todetermine if a gap in the analysis of sentiment exists requiring furtherprocessing to improve the analysis of sentiment, wherein the gap existsin sentiment analysis of source materials determined to be insufficientto derive sentiment for aspects of entities, and wherein gapidentification uses the policies to identify the gap based on measuresof sentiment not meeting one or more thresholds in the policies;responsive to the computer analyzing the data and the analysis ofsentiment to determine if the gap in the analysis of sentiment exists,determining, by the computer, an amount of positive sentiment in thedata and negative sentiment in the data for each topic in a set oftopics corresponding to aspects of an entity and selecting, by thecomputer, a topic from the set of topics corresponding to a particularaspect of the entity for the further processing based on determiningthat the amount of positive sentiment in the data and negative sentimentin the data for the selected topic fails to meet a threshold amount ofpositive and negative sentiment for the selected topic to derivesentiment for the particular aspect of the entity; generating, by thecomputer, a task using a particular task template, in a set of tasktemplates, matching a type of the gap to address the gap in the analysisof sentiment regarding the selected topic that failed to meet thethreshold amount of positive and negative sentiment for the selectedtopic to derive sentiment for the particular aspect of the entity,wherein the task comprises a specification for performing the taskcomprising a description of activity, a domain ontology, a descriptionof the gap, a description of the task, sentiment information, measuresof sentiment about aspects of entities associated with the gap,questions for a survey from the task template, and instructions forcompleting the task to address the gap; and using, by the computer,crowdsourcing to submit the task for processing to address the gap inthe analysis of sentiment regarding the selected topic that failed tomeet the threshold amount of positive and negative sentiment for theselected topic to derive sentiment for the particular aspect of theentity, wherein the computer using the crowdsourcing to submit the taskfor processing further comprises: identifying, by the computer, usersassociated with the topic about the particular aspect of the entitybased on an amount of sentiment information about the topic in retrieveddata from each user; determining, by the computer, a group sentimentabout the topic corresponding to a group of users on a social network;and selecting, by the computer, one or more users within the group ofusers on the social network to submit the task to based on thedetermined group sentiment about the topic.
 18. The computer programproduct of claim 17, further comprising: re-calculating, by thecomputer, the analysis of sentiment based on receiving a result of theprocessing of the task to address the gap in the analysis of sentimentregarding the selected topic that failed to meet the threshold amount ofpositive and negative sentiment for the selected topic to derivesentiment for the particular aspect of the entity; and determining, bythe computer, that the received result of the processing of the taskclosed the gap in the analysis of sentiment regarding the selected topicbased on the re-calculating of the analysis of sentiment in response toa policy in the policies that previously identified the gap no longeridentifying the gap.
 19. The computer program product of claim 17,wherein the analyzing of the sentiment is for the aspects of the entitycorresponding to an activity, and wherein the gap in the analysis ofsentiment requiring the further processing corresponds to the topicabout the particular aspect of the entity.
 20. The computer programproduct of claim 19, and further comprising: identifying, by thecomputer, the topic about the particular aspect of the entity based onsentiment information about the topic in the data.
 21. The computerprogram product of claim 19, wherein the task generated by the computercomprises a specification for addressing the gap based on the particulartask template corresponding to the type of gap of the gap.